Papers with multilingual settings
Copied to clipboard
| Challenge: | Reward models (RMs) are primarily trained and evaluated in English and their capabilities in multilingual settings remain understudied. |
| Approach: | They construct a multilingual RM evaluation benchmark that tests the chat, safety, reasoning, and translation capabilities of RMs in 23 languages. |
| Outcome: | The proposed model performs better for high-resource languages and improves with translation quality. |
Copied to clipboard
| Challenge: | Large language models (LLMs) remain unreliable for global enterprise applications due to performance gaps between high-resource and mid/low-resourced languages . |
| Approach: | They propose a batch-wise alignment strategy that aligns model outputs across languages . this method improves non-English accuracy by up to 23.9% without compromising English performance . |
| Outcome: | The proposed approach improves non-English accuracy by up to 23.9% without compromising English performance, model reasoning, or retrieval quality. |
Copied to clipboard
| Challenge: | Concept-to-text Natural Language Generation requires a delexicalisation of the input, but this often requires that the input appears verbatim in the output text. |
| Approach: | They propose a method that uses multilingual pretrained embeddings to inflect words in their correct form during relexicalisation. |
| Outcome: | The proposed method outperforms monolingual models in concept-to-text and in low resource environments. |
Copied to clipboard
| Challenge: | Existing studies evaluate whether large language models handle global cultural diversity . however, mechanisms behind cultural knowledge acquisition remain unexplored . |
| Approach: | They propose an interpretable framework to study cultural knowledge transfer in large language models . they observe bidirectional cultural transfer between English and other high-resource languages . |
| Outcome: | The proposed framework ensures training data transparency and controls transfer effects. |
Copied to clipboard
| Challenge: | We present multilingual parallel template filling datasets for MUCs . systems were required to extract one template per incident, containing details about perpetrators, victims, weapons used . |
| Approach: | They introduce MultiMUC, the first multilingual parallel corpus for template filling . they obtain automatic translations from a strong multilingual machine translation system . |
| Outcome: | The proposed dataset includes translations of the classic MUC-4 template filling benchmark into Arabic, Chinese, Farsi, Korean, and Russian. |
Copied to clipboard
| Challenge: | Existing benchmarks and resources for evaluating gender biases in multilingual settings are limited. |
| Approach: | They propose to extend DisCo to different Indian languages using human annotations to evaluate gender biases in multilingual models. |
| Outcome: | The proposed benchmarks and mitigation techniques are extended beyond English to evaluate gender biases in multilingual models. |
Copied to clipboard
| Challenge: | Large language models (LLMs) can be benchmark-contaminated, resulting in inflated scores that mask memorization as generalization. |
| Approach: | They use the FLORES-200 translation benchmark as a diagnostic to investigate cross-direction data contamination. |
| Outcome: | The proposed model can be cross-directional, boosting performance in unseen translation directions due to target-side memorization. |
Copied to clipboard
| Challenge: | Large language models are increasingly deployed in multilingual settings that process sensitive data . prior privacy evaluations focused on English, but new research shows that language matters for privacy leakage . |
| Approach: | They quantify six corpus-level linguistic indicators and evaluate vulnerability under three attack families. |
| Outcome: | The results show that language matters for privacy leakage in large language models . Italian exhibits the strongest exposure, while English and French are more resilient . |
Copied to clipboard
| Challenge: | Pretrained language models are integral part of AI applications, but their high computational cost limits accessibility. |
| Approach: | They evaluate Aurora-M, a 15B parameter multilingual open-source model trained on English, Finnish, Hindi, Japanese, Vietnamese, and code. |
| Outcome: | The proposed model outperforms existing models on English, Finnish, Hindi, Japanese, Vietnamese, and code. |
Copied to clipboard
| Challenge: | Large Language Models excel at a low-resource level given limited data, but are unsuitable for runtime systems which require low latency. |
| Approach: | They propose a method to augment training data for a model 40x smaller (500M parameters) they use Alexa to generate synthetic data from Alexa 20B to augment the training set . |
| Outcome: | The proposed method improves low-resource SP on two datasets in low-source settings. |
Copied to clipboard
| Challenge: | Several modifications have been proposed to improve monolingual language models, but none of them result in better multilingual models. |
| Approach: | They propose to add positional encodings to token embeddings to preserve word-order information in a non-autoregressive setting. |
| Outcome: | The proposed modifications tend to improve monolingual models, but none improve multilingual models. |
Copied to clipboard
| Challenge: | Current LLMs are primarily trained on English data but also include data from other languages. |
| Approach: | They propose to use a pre-translation strategy to translate a task prompt into English before inference . they use 'a modular entity' that could be translated into four different languages . |
| Outcome: | The proposed strategies are based on a set of pre-trained data across 35 languages covering both low and high-resource languages. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly deployed in multilingual settings . but most bias evaluation remains English-centric and ignores how bias manifests within reasoning . |
| Approach: | They evaluate large language models with supervised fine-tuning and preference optimization . they find that bias varies substantially across languages, with consistent degradation in non-English settings . |
| Outcome: | The proposed model improves in English, Dutch, Spanish, and Turkish using the MBBQ benchmark. |
Copied to clipboard
| Challenge: | Existing approaches to train a multilingual NMT model for low-resource languages are lacking in terms of number of supervised examples. |
| Approach: | They propose to use decoder pre-training and back-translation to solve the degeneracy problem by analyzing spurious correlations between source and decoded sentences. |
| Outcome: | The proposed methods show significant improvement over the pivot-based approach on three challenging multilingual datasets. |
Copied to clipboard
| Challenge: | Recent studies show that malicious prompt instructions could solicit objectionable content from LLMs. |
| Approach: | They compare how state-of-the-art LLMs respond to malicious prompts in different languages . they find that LLM's generate unsafe responses more often when a prompt is written in a lower-resource language . |
| Outcome: | The proposed model can generate unsafe responses more often when a malicious prompt is written in a lower-resource language, and less irrelevant responses when written in lower-source languages. |
Copied to clipboard
| Challenge: | Existing safeguards for Large Language Models are vulnerable to "jailbreaking" harmful queries. |
| Approach: | They propose a learning method that optimizes a continuous soft safety prompt automatically to facilitate multilingual safeguarding of LLMs. |
| Outcome: | The proposed method outperforms previous approaches in multilingual jailbreak defense while exhibiting strong cross-lingual generalization. |
Copied to clipboard
| Challenge: | Relation extraction (RE) is a fundamental task in information extraction, but its extension to multilingual settings is hindered by the lack of supervised resources comparable in size to large English datasets. |
| Approach: | They propose a dataset to analyze relation extraction (RE) in multilingual settings . they find machine translation is a viable strategy to transfer RE instances . |
| Outcome: | The proposed dataset covers 12 typologically diverse languages from 9 language families and is compared with existing datasets. |
Copied to clipboard
| Challenge: | Existing studies on LLM abstention focus on English, but they show that it can reduce the accuracy of the model by 20.5% . |
| Approach: | They propose to teach LLMs to abstain in the face of knowledge gaps by generating multiple feedback items in related languages. |
| Outcome: | Extensive experiments show that the proposed approach outperforms baselines and achieves 9.2% improvement for low-resource languages. |
Copied to clipboard
| Challenge: | Recent large language models demonstrate remarkable abilities in responding to queries in diverse languages, but their ability to handle long multilingual contexts is unexplored. |
| Approach: | They propose a multilingual Needle-in-a-Haystack (MLNeedle) test to assess a model's ability to retrieve relevant information from a collection of multilingual distractor texts. |
| Outcome: | The proposed model performance is the lowest when the needle is in a language outside the English language family and (ii) located in the middle of the input context. |
Copied to clipboard
| Challenge: | Character-level models of tokens are effective at dealing with within-token noise and out-of-vocabulary words. |
| Approach: | They propose to eliminate the need for tokenizers by using a character-level semi-Markov conditional random field that uses neural networks for its character and segment representations. |
| Outcome: | The proposed model outperforms state-of-the-art part-of speech taggers on a noisy English dataset. |
Copied to clipboard
| Challenge: | Existing approaches to evaluate image captions are English-centric, despite improvements in the CLIPScore metric . however, there are no available benchmarks for multilingual captioning evaluation . |
| Approach: | They propose to use machine-translated and machine-repurposed datasets to evaluate CLIPScore variants in multilingual settings. |
| Outcome: | The proposed evaluation strategies are based on machine-translated and human judgements. |
Copied to clipboard
| Challenge: | Existing multilingual long-context benchmarks are myopic and inherently limited, as successful recall alone does not indicate a model’s capacity to reason over extended contexts. |
| Approach: | They propose a new synthetic benchmark for multilingual long-context reasoning that includes bAbI-style tasks that test multi-hop inference, aggregation, and epistemic reasoning. |
| Outcome: | The proposed benchmarks are based on a multilingual long-context model and span seven languages. |
Copied to clipboard
| Challenge: | Semantic parsers treat graphs as strings or trees, but there is no guarantee that the output is a well-formed graph. |
| Approach: | They propose a graph-aware sequence model that utilizes a DAG grammar to guide graph generation. |
| Outcome: | The proposed model outperforms string-based and DAG-grammar models by a large margin and can guarantee the well-formed graphs. |
Copied to clipboard
| Challenge: | Under-resourced and endangered or small languages yield problems for automatic processing and exploiting because of the small amount of available data. |
| Approach: | They propose an approach using enriched linguistic research data to create communication boards commonly used in alternative augmentative communication (AAC) using lexical analysis and rich annotation, the boards can be imported into various AAC software. |
| Outcome: | The proposed approach uses lexical analysis and rich annotations to create communication boards commonly used in alternative augmentative communication (AAC) The created boards can be imported into various AAC software and are available under the CC BY-NC-SA 4.0 (public) license. |
Copied to clipboard
| Challenge: | Existing approaches to improve retrieval accuracy and generation quality of large language models suffer from language preference. |
| Approach: | They propose a framework that explicitly disentangles multilingual RAG into language-controllable retrieval and language-agnostic reasoning. |
| Outcome: | Experimental results show that the proposed approach outperforms baselines across multilingual benchmarks. |
Copied to clipboard
| Challenge: | Recent studies have introduced eclectic strategies to enhance MLLMs’ reasoning capabilities, but they remain related to a single language. |
| Approach: | They propose a modular approach that instructs models to abstract key elements of the reasoning process and refine reasoning trajectories via self-correction. |
| Outcome: | The proposed approach improves multimodal reasoning, gets aligned performances among the languages approaching strong models and improves the model's performance. |
Copied to clipboard
| Challenge: | Existing efforts to distill reasoning capabilities have focused mainly on English, leaving multilingual distillation underexplored. |
| Approach: | They propose a method that incorporates incorrect rationales as additional guidance to improve multilingual reasoning in large language models. |
| Outcome: | Empirical results show that d-CoT-nR significantly surpasses the baseline, improving accuracy in unseen languages and correctness in step-by-step reasoning. |
Copied to clipboard
| Challenge: | Existing safety standards are often based on direct translations from English, which overlook key aspects of local communication. |
| Approach: | They propose a high-quality, human-verified safety evaluation dataset tailored for the Indonesian context. |
| Outcome: | The proposed dataset covers formal and colloquial Indonesian, along with three major local languages: Javanese, Sundanese, and Minangkabau. |
Copied to clipboard
| Challenge: | Using a neural reranking model, we can match a UMLS concept with a Wikipedia page, enabling manual alignment with minimal effort. |
| Approach: | They propose a cross-lingual neural reranking model to match a UMLS concept with a Wikipedia page, which achieves a recall@1of 72%, a substantial improvement of 20% over word- and char-level BM25. |
| Outcome: | The proposed model achieves recall@1of 72%, 20% better than word- and char-level BM25, and will facilitate easier access to Wikipedia for health professionals, patients, and NLP systems, including in multilingual settings. |
Copied to clipboard
| Challenge: | Large Language Models reproduce and exacerbate social biases present in training data, and resources to quantify this issue are limited. |
| Approach: | They propose a multilingual parallel dataset to examine culturally-specific stereotypes that may be learned by LLMs. |
| Outcome: | The proposed dataset includes stereotypes from 20 regions around the world and 16 languages, spanning multiple identity categories subject to discrimination worldwide. |
Copied to clipboard
| Challenge: | Large language models (LLMs) have revolutionized natural language processing and are limited by high inference time in multilingual settings. |
| Approach: | They propose a training recipe for an assistant model in speculative decoding, which are leveraged to draft and-then its future tokens are verified by the target LLM. |
| Outcome: | The proposed model significantly speeds up inference time and out-of-domain speedup across various languages. |
Copied to clipboard
| Challenge: | Existing hallucination detection benchmarks operate at the sequence level and are limited to English . Existing methods lacking fine-grained, multilingual supervision are limited in English based on the sequence . |
| Approach: | They propose a large-scale, multilingual dataset annotated with span-level hallucinations across 14 languages. |
| Outcome: | The proposed dataset annotated with span-level hallucinations across 14 languages is scalable and cost-efficient. |
Copied to clipboard
| Challenge: | Winograd schemas are well-established tools for evaluating coreference resolution and commonsense reasoning capabilities of computational models. |
| Approach: | They present a dataset of German, French, and Russian schemas aligned with their English counterparts. |
| Outcome: | The proposed model improves in English and German, while the model improve in other languages. |
Copied to clipboard
| Challenge: | Existing methods to reduce hallucinations in large language models are inaccurate and inaccuracies in the generated feedback. |
| Approach: | They propose a method that helps LLMs determine whether to utilize multiple generated feedback responses and how to identify the most useful ones. |
| Outcome: | Extensive experiments show that the proposed method outperforms baselines on encyclopedic and commonsense knowledge QA tasks. |
Copied to clipboard
| Challenge: | Existing V&L pre-training methods rely on strictly-aligned multilingual image-text pairs generated from English-centric datasets. |
| Approach: | They propose a regularized cross-lingual visual contrastive learning objective that constrains representation proximity of weakly-aligned multilingual image-text pairs. |
| Outcome: | The proposed model outperforms competing models with weak zero-shot capability on 5 multi-modal tasks across 6 languages. |
Copied to clipboard
| Challenge: | Existing evaluations of LLMs in finance are text-only, monolingual, and largely saturated by current models. |
| Approach: | They propose a multilingual and multimodal benchmark for evaluating LLMs in real financial contexts. |
| Outcome: | The first expert-annotated multilingual and multimodal benchmark is released . it evaluates 21 leading LLMs and shows they perform better in multilingual settings . |
Copied to clipboard
| Challenge: | a global issue that shapes public discourse shapes opinion and decision-making . many multilingual work has focused on claim verification rather than generating explanatory verdicts . |
| Approach: | They propose a multilingual dataset designed for verdict generation covering eight European languages. |
| Outcome: | The EuroVerdict dataset covers claims, manual verdicts, and supporting evidence . it is compared with other datasets in eight European languages . |
Copied to clipboard
| Challenge: | Existing studies on the use of Large Language Models (LLMs) focus primarily on English, leaving the cross-lingual generalization of aligned behavior underexplored. |
| Approach: | They propose a structured generator-extractor pipeline and a multi-dimensional distributional analysis framework to examine how narrative attributes vary across languages, models, and social conditions. |
| Outcome: | The proposed model reveals substantial cross-lingual variability in narrative generation patterns, indicating that distributions observed in English do not always exhibit similar characteristics in other languages, particularly in lower-resource settings. |
Copied to clipboard
| Challenge: | Large language models (LLMs) can explain grammatical rules, but fail to apply those rules when judging sentence acceptability. |
| Approach: | They propose a grammar prompting paradigm that feeds an LLM's metalinguistic explanation back to the target model before deciding which sentence of a minimal pair is grammatical. |
| Outcome: | The proposed model improves on the English BLiMP, Chinese SLING, and Russian RuBLimp benchmarks. |
Copied to clipboard
| Challenge: | Existing approaches address key factors that influence multilingual ICL, but they do not integrate them into the model. |
| Approach: | They propose a method that quantifies and optimally balances three factors for improved example selection. |
| Outcome: | Experiments on mCSQA and TYDI show that the proposed method outperforms existing methods. |
Copied to clipboard
| Challenge: | Negations are key to determining sentence meaning, making them essential for logical reasoning. |
| Approach: | They construct and publish two new textual entailment datasets in four languages with paired examples differing in negation. |
| Outcome: | The results show that increasing the model size may improve the models’ ability to handle negations. |
Copied to clipboard
| Challenge: | Existing benchmarks for numerical reasoning in multilingual Indic languages are inadequate . e.g., FinVQA is a framework for evaluating financial numerical reasoning . |
| Approach: | They propose a framework that combines supervised fine-tuning with constraint-aware decoding to promote faithful numerical reasoning. |
| Outcome: | The proposed framework spans English, Hindi, Bengali, Marathi, Gujarati, and Tamil . it combines supervised fine-tuning with constraint-aware decoding to promote faithful numerical reasoning . |
Copied to clipboard
| Challenge: | Existing methods for dense retrieval in multilingual environments encode language identity alongside semantics. |
| Approach: | They propose a method that trains on pooled embeddings to remove language-identity signal directly in vector space. |
| Outcome: | The proposed method improves ranking quality and cross-language coverage across multiple languages with especially strong gains for script-distinct languages. |
Copied to clipboard
| Challenge: | Large companies and services handle a substantial volume of reviews and social media mentions. |
| Approach: | They propose a generalizable pipeline for automated customer feedback analysis using backtranslation and fine-tuned models to generate 1.7 million tweets from a small seed corpus, complemented by synthetic reasoning traces. |
| Outcome: | The proposed pipeline generates 1.7 million synthetic tweets from a small seed corpus, complemented by synthetic reasoning traces. |
Copied to clipboard
| Challenge: | Subword tokenization schemes such as Byte Pair Encoding (BPE) are widely adopted, but their effectiveness in multilingual settings remains understudied. |
| Approach: | They propose a multilingual tokenizer that produces linguistically coherent tokens for multilingual LLMs. |
| Outcome: | The proposed tokenizer improves fertility score by 39.5% over LLaMA4 and 18% over Sutra. |